Use Routers for Load Balancing
Instead of using load balancer to load balancing a large volume of traffic, routers are much cheaper of doing so.
The traditional solution with load balancers looks like this: Use Load Balancers for distributing traffic
Routers can be used as load balancers by its equal-cost multi-path (ECMP) forwarding for destinations.1
- Name an IP as the service IP. this IP should not exist in the network, or this method will create a black hole.
- Redirect all traffic to the routers for load balancing
- On the routers, use policy/static routes to destinate the incoming traffic to servers behind
- The servers can use any IPs as long as they can communicate with the routers
On the servers, use the iptables to change destination IP for incoming packets and source IP for replies:2
incoming
iptables -t nat -A PREROUTING -d <service IP> -j DNAT --to-destination <server IP:port>
outgoing
iptables -t nat -A POSTROUTING -d <service IP:port> -j SNAT --to-source <server IP>
There is one issue: how to hold states of connections (TCP, TLS and etc.)
The routers have different hash combinations to determine which path should be use for equally distribution. Here we can set it to source IP + source port + destination IP + destination port. Then each connection will always be handled by one server.
Connection ID in the HTTP/3.0 (QUIC) is completely ignored by this method,3 causing communication errors in the latest protocol (HTTP/3.0)